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Decomposition  Techniques  for  Temporal 
Resource  Allocation 


Cynara  C.  Wu  and  David  A.  Castanon 


Abstract — ^We  consider  the  problem  of  allocating  a  set  of 
heterogeneous  resources  with  availability  constraints  to 
maximize  a  given  value  function.  The  problem  arises  in  a  wide 
variety  of  military  and  industrial  situations.  We  formulate  the 
problem  as  a  discrete-state  decision  process.  We  consider  two 
instances  of  the  problem  that  is  applicable  in  situations  where 
persistent  coverage  over  all  stages  is  desired.  While  we  were 
able  to  solve  the  first  example  using  dynamic  programming, 
the  computational  requirements  are  significant  and  not 
scalable  to  larger  instances.  We  consider  an  approximate 
technique  using  decomposition  combined  with  dynamic 
programming.  Our  experiments  show  that  this  approach 
requires  very  little  computation  time  and  produces  near- 
optimal  results  for  the  examples  considered. 

I.  Introduction 

HE  allocation  over  time  of  heterogeneous  resources 
with  availability  constraints  is  a  problem  that  has 
applications  in  manufacturing,  telecommunications,  the 
military,  and  other  areas.  Examples  include  allocating 
different  machines  with  require  ddWntimeSTb  jobs,  as  well 
as  allocating  vehicles  with  various  capabilities  and  fuel 
limitations  to  different  geographic  areas  to  perform  tasks. 
The  majority  of  such  problems  are  known  to  be  NP-hard 
and  numerous  approximation  techniques  have  been 
developed  to  solve  them. 

We  consider  an  approximate  solution  based  on 
decomposition.  Decomposition  approaches  have  been  used 
to  approximately  solve  a  variety  of  hard  combinatorial 
problems,  including  resource  allocation  and  scheduling 
problems.  They  have  been  combined  in  a  straightforward 
manner  with  sub-gradient  and  cutting-plane  methods  to 
solve  simultaneous  routing  and  resource  allocation  in 
wireless  networks  [3].  They  have  been  combined  with 
feature  extraction  for  admission  control  in  wireless 
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networks  [5],  They  have  been  used  to  schedule  test 
operations  by  decomposing  the  problem  according  to 
available  work  centers  [4].  Work  has  also  been  done  in 
developing  novel  heuristics  for  decomposing  problems  such 
as  the  Constraint  Satisfaction  Problem  [2].  We  present  a 
straightforward  approach  based  on  decomposing  the 
problem  according  to  the  available  resources  and  solving 
the  sub-problems  using  dynamic  programming.  While  this 
approach  has  very  likely  been  applied  to  combinatorial 
problems,  we  are  unaware  of  specific  applications  based  on 
dynamic  programming  for  resource  allocation  problems. 

The  paper  is  organized  as  follows.  In  Section  n,  we 
formulate  the  problem.  In  Section  HI,  we  describe  the 
dynamic  programming  solution  to  the  problem.  In  Section 
rv,  we  describe  the  approximate  solution  based  on 
decomposition.  In  Sections  V  and  VI,  we  illustrate  the 
solutions  using  two  examples  and  present  some 
computational  results. 

n.  Problem  Formulation 

We  assume  a  set  of  R  resources  that  are  to  be  allocated  to 
M  groups  over  N  stages.  Resource  i  is  available  for  up  to  t^ 
consecutive  stages,  at  which  point  it  is  not  available  again 
until  a  minimum  of  s,-  consecutive  stages  has  passed.  Fig.  1 
provides  a  sample  allocation  schedule. _ 


|u/a|u/a|  1  1  1  1  1  1  2  1  2 

1  |u/a|u/a|  2  1  1 

*  Max  #  consecutive 

available  stages  s 

Min  #  conse¬ 
cutive  unavail¬ 
able  stages  =  Sf 

Figure  1  Allocation  schedule  where  resource  i  is 
unavailable  for  4  stages,  and  allocated  to  groups  1  and  2 
for  5  and  3  stages,  respectively. 

The  objective  is  to  allocate  resources  subject  to  each 
resource’s  availability  constraints  to  maximize  a  given 
value  function.  We  assume  that  the  value  function 
accumulates  over  time;  i.e.,  the  value  function  is  the  sum  of 
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values  attained  during  each  stage  and  these  values  depend 
only  on  the  states  and  decisions  over  each  of  the  stages.  We 
also  assume  that  there  is  no  cost  in  changing  the  allocation 
of  one  resource  from  one  group  to  another  between  stages. 

This  problem  can  be  formulated  as  a  deterministic  finite- 
horizon,  discrete-state  decision  process.  The  state  at  stage  k 
is  represented  by  a  vector  where 

x^i  is  an  ordered  pair  that  provides  the  availability  and 
group  information  for  resource  i  at  stage  k.  Specifically,  if 
resource  i  has  been  made  available  for  stages  k-n  through  k, 
was  not  available  at  stage  k-n-1,  and  has  been  allocated  to 
group  m  during  stage  k,  then  we  have 
I**  addition,  if  resource  i  has  been 
made  unavailable  for  stages  k-n  through  k  and  was  available 
at  stage  k-n-1,  then  we  havej:^;  =  (min(0,n-j,  ),0).  In  the 

former  case,  the  first  element  of  vector  is  the  number  of 
stages  for  which  resource  i  has  been  made  available.  In  the 
latter  case,  the  first  element  of  vector  is  the  number  of 
stages  until  resource  i  will  again  be  available. 

The  decision  vector  f/*  ^  indicates 

which  resources  will  be  available  to  which  groups  during 
stage  J(:+ 1 ,  In  particular,  we  have 

,  ,  if  resource  i  is  to  be  available  to 

M  2 1  ’  group  mat  stage /:  +  l, 

^ki  —  v*fa'  >  ^ki  I  ~  ' 

(0,0)  Otherwise. 

The  possible  choices  for  each  decision  component  depend 
on  the  state  at  stage  k.  At  any  state,  the  decisions  for  any 
resource  include  making  the  resource  unavailable  for  the 
next  stage.  For  states  where  xj[,  <  tj ,  the  decisions  also 
include  allocating  the  resource  to  any  of  the  M  groups  for 
the  next  stage. 

The  state  for  each  individual  resource  evolves  as  follows: 
-  fki^k^^k) 

(xy  -Hl,m)  if  Uti  =  (l,m),  me  {l,2,...,Af]; 
f-  Sj  + 1,0)  if  «*,■  =  0  and  xh  ^  0, 

U-  +  l.o)  if  "ki  =  0  and  xl  =  0. 

Given  a  set  of  value  functions  g*,  the  optimal  set  of 

decisions  is  that  which  maximizes  the  value: 

N-l 

gNiXNl  +  'ZSkiXkrU,). 

*=0 

Note  that  this  problem  can  also  be  formulated  as  a 
general  integer  programming  problem: 


maxg(x) 
subject  to  Ax  <b 

where  =  1  when  resource  i  has  been  allocated  to  group  j 

during  stage  k  and  where  A  imposes  the  availability 
constraints  for  the  resources.  One  possible  approach  to 
solving  this  problem  is  to  enumerate  aJl  of  the  combinations 
of  possible  decision  sets.  However,  the  number  of 

combinations  is  approximately  ((M  +  1)*)^.  Although  the 
dynamic  programming  solution  described  below  is  also 
exponential,  its  computational  requirements  are 
significantly  smaller. 

m.  DYNAMIC  Programming  Solution 

The  decision  process  formulated  in  the  previous  section 
can  be  solved  exactly  using  dynamic  programming.  (See  [1] 
for  a  detailed  treatment  of  this  topic.)  Let  denote 

the  optimal  value-to-go,  or  value  that  can  be  attained 
starting  at  state  at  stage  jfc.  The  value  for  every  possible 
state  for  the  last  stage  is  simply  /^(X;v)  =  g;v(X^). 
Assuming  we  have  the  value-to-go  for  every  possible  state 
for  the  (it+l)st  stage,  we  can  determine  the  optimal 
decisions  and  the  associated  values-to-go  for  each  possible 
state  for  the  kth  stage  as  follows.  For  each  state,  we 
consider  each  possible  decision  and  compute  the  sum  of  the 
value  attained  over  the  current  stage  under  the  given 
decision  and  the  value-to-go  for  the  resulting  state  for  stage 
k+1.  The  optimal  decision  and  associated  value-to-go  for 
the  state  is  that  with  the  maximum  sum: 

A  (X*)  =  max{g*  (X* ,  [/* )  -H  7*^,  {/*  (X*  ,17* )} . 

"k 

At  any  stage,  each  resource  can  be  in  one  of 
Sf  unavailable  states  (since  any  state  in  which  a  resource 
has  been  made  unavailable  for  more  than  Sj  stages  is 
equivalent  to  the  state  in  which  the  resource  has  been  made 
unavailable  for  exactly  s,-  stages),  or  in  one  ofAfr,-  available 
states.  The  number  of  states  in  this  problem  for  a  particular 

R 

stage  is  therefore  J~J(Mt,  +s,) .  For  any  state,  the  number 
(=1 

of  possible  decisions  is  up  to  {M  -I- 1)* ,  where  the  maximum 
number  of  decisions  occurs  when  all  resources  can  be  made 
available  for  the  next  stage.  The  total  number  of  decisions 
to  evaluate  to  compute  the  optimal  solution  is  then 
approximately 

f  R  'j 

J|(Mr,  +s,.)  (M-t-lfN.  (1) 

V  1=1  y 

While  complete  enumeration  is  exponential  in  both  the 
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number  of  resources  and  the  number  of  stages,  this 
approach  is  only  exponential  in  the  number  of  resources. 
However,  using  dynamic  programming  to  solve  such 
problems  exactly  is  still  impractical  for  problems  with  a 
large  number  of  resources,  particularly  if  the  problems  need 
to  be  solved  in  real-time. 

rv.  Decomposition  Approach 

Since  the  computation  time  for  obtaining  an  exact 
solution  is  exponential  in  the  number  of  resources,  a 
straightforward  approach  to  addressing  the  computational 
limitations  of  exact  dynamic  programming  is  to  decompose 
the  problem  into  sub-problems  associated  with  the 
individual  resources.  One  application  of  this  approach  is  to 
consider  each  resource  in  turn  and  select  an  allocation 
schedule  for  that  resource  by  applying  dynamic 
programming  to  determine  the  optimal  solution  for  that 
individual  resource  assuming  that  the  schedules  for 
previously  assigned  resources  are  fixed  and  that  there  are  no 
additional  resources  to  allocate. 

Let  0(i)  provide  the  ordering  in  which  the  resources  are 
to  be  considered.  We  first  consider  resource  (5(1) ,  assume 
that  all  remaining  resources  are  never  made  available,  and 
determine  the  allocation  schedule 

that  maximizes  the  value  function 

N-\ 

This  allocation  is  then 

*=0 

assigned  to  resource  (5(1) .  For  each  subsequent 
resource  <5(0 ,  we  assume  that  the  schedules  for 
resources (5(1), (5(2),..., (5(i-l)  are  set  to  those  previously 
computed  and  that  the  schedules  for  resources 
(5(0,(5(i+l),...,(5(M)  are  never  made  available,  and 

determine  the  allocation  schedule  (^io(,)»^20(i)’"-’-*/vo(i)) 
that  maximizes  the  value  function 

N~\ 

^v(^v)+2]^i(^t>^*).  A.  general  decomposition 

k=0 

approach  in  which  the  sub-problems  can  be  solved  using 
any  algorithm  is  illustrated  in  Fig.  1. 


Figure  2  Illustration  of  a  decomposition  approach.  Each 
resource,  represented  here  as  cylinders,  is  considered  in 
some  given  order  and  allocated  one  of  its  feasible 
schedules.  During  the  allocation  of  its  schedule,  two 
assumptions  are  made.  First,  remaining  resources  are 
allocated  some  pre-determined  schedule,  which  in  this 
example  is  one  in  which  they  are  never  allocated. 
Second,  resources  thatliave  already  been  considered  are 
flxed  at  the  schedules  previously  determined.  Any 
algorithm  can  be  used  to  determine  which  of  its  feasible 
schedules  to  allocate  to  a  resource.  Our  proposed 
approach  uses  dynamic  programming  to  determine  the 
optimal  schedule  given  the  two  assumptions. 

The  number  of  states  for  the  sub-problem  associated  with 
resource  i  is  Mtj  +  s, .  For  each  of  these  states,  the  number 
of  possible  decisions  is  either  I  or  M+1.  The  number  of 
decisions  to  evaluate  to  compute  the  optimal  solution  of  the 
sub-problem  associated  with  resource  i  is  then  bounded  by 
{Mtj  +  Sj  XM  -f-  l)N ,  resulting  in  the  number  of  decisions  to 

evaluate  for  the  entire  problem  of 

R 

2][{A/r,+s,.XM-H);v].  (2) 

1=1 

Since  the  allocations  of  resources  that  have  not  been 
assigned  are  not  considered  during  the  scheduling  of  a 
particular  resource,  the  basic  decomposition  approach 
described  above  is  a  greedy  algorithm.  In  fact,  this 
approach  is  an  example  of  an  approximate  dynamic 
programming  algorithm  where  the  value-to-go  for 
unassigned  resources  is  given  an  estimate  of  zero.  As  will 
be  seen  in  the  example  described  in  the  following  section, 
this  approach  can  obtain  solutions  that  are  very  close  to  the 
optimal.  However,  for  certain  value  functions,  approximate 
dynamic  programming  approaches  that  provide  more 
accurate  estimates  of  the  value-to-go  are  likely  to  obtain 
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improved  results.  We  are  currently  considering  more 
complex  value  functions  in  order  to  evaluate  additional 
approximate  approaches. 

V.  Example  l 

The  following  example  is  motivated  by  a  problem  where 
the  main  objective  is  to  ensure  that  surveillance  over  a 
single  defined  area  of  interest  can  be  provided  over  an 
entire  scenario. 

A.  Example  Details 

In  this  example,  the  objective  is  to  allocate  23  vehicles 
over  36  stages  to  a  single  area  of  interest.  The  vehicles  are 
of  4  different  types,  each  of  which  has  a  surveillance  range 
that  is  some  Auction  of  the  total  area  of  interest.  Vehicles 
carry  a  certain  amount  of  fuel  which  allows  them  to  patrol 
in  the  area  of  interest  for  a  certain  amount  of  time.  When 
their  fuel  is  low,  they  require  a  certain  amount  of  downtime 
in  order  to  travel  to  a  station  to  refuel.  The  objective  is  to 
determine  a  schedule  to  indicate  which  vehicles  are 
available  during  which  stages  that  maximizes  a  given  value 
function. 

The  following  table  provides  for  each  vehicle  type  the 
number  of  available  vehicles,  the  maximum  number  of 
consecutive  stages  each  vehicle  type  is  available  before  it 
needs  to  refuel,  the  minimum  number  of  consecutive  stages 
each  vehicle  type  is  unavailable  after  it  heads  to  a  station  to 
refuel,  and  the  fiaction  of  the  total  area  of  interest  for  which 
each  vehicle  type  can  provide  surveillance  coverage. 


Table  1  Resource  Data  for  Example  1 


Vehicle 

Type 

Number 

Max 

Availability 

Min 

Off-Time 

Coverage 

Provided 

1 

9 

60 

1 

1/240 

2 

1 

13 

1 

1 

3 

4 

1 

1 

0.5 

4 

9 

2 

1 

0.125 

Since  the  objective  of  the  problem  is  to  provide  sufficient 
surveillance  at  all  times,  we  selected  a  value  function  that  is 
independent  of  the  stage  and  is  monotonically  non¬ 
decreasing  in  the  total  amount  of  coverage  that  could  be 
provided  by  the  vehicles  allocated  to  a  stage.  In  addition, 
the  slope  of  the  value  function  is  greatest  when  the  fraction 
of  the  coverage  is  less  than  1  and  decreases  again  when  the 
fraction  of  coverage  is  greater  than  1.5  and  greater  than  3. 
The  decrease  in  slope  as  the  fraction  of  coverage  increases 
represents  the  decrease  of  importance  of  additional 
coverage  when  certain  amounts  of  coverage  have  been 
reached.  The  function  we  selected  is  illustrated  in  the  Fig.  2. 


Figure  3  Value  function  for  Example  1 


B.  Computational  Results 


Note  that  vehicles  of  type  1  can  be  made  available  for  up 
to  60  stages.  Since  this  is  greater  than  the  length  of  the 
scenario  being  considered  and  since  the  value  function  is 
monotonically  non-decreasing  in  the  amount  of  coverage 
available  at  any  stage,  any  optimal  solution  would  make 
these  vehicles  available  during  each  stage.  Note  also  that 
the  vehicle  of  type  2  has  such  a  large  surveillance  range  that 
it  can  be  shown  that  any  optimal  solution  would  limit  the 
total  number  of  unavailable  stages  to  2.  Since  the  value 
function  is  uniform  over  all  stages,  it  is  straightforward  to 
select  a  schedule  for  the  vehicle  of  type  2  that  would  result 
in  the  optimal  schedule.  Therefore,  finding  the  optimal 
solution  for  this  example  reduces  to  finding  the  optimal 
schedule  for  vehicles  of  type  3  and  4. 

Using  (1),  we  see  that  to  determine  the  optimal  allocation 
of  vehicles  of  type  3  and  4  using  dynamic  programming,  the 
number  of  decisions  that  need  to  be  evaluated  is 

(r,.  -I-  )  (2f  N  =  (2“  •  3^  )(2)'^  (36) «  90.7  billion. 

V  /=i 

Using  (2),  we  see  that  to  determine  the  allocation  of 
vehicles  of  type  3  and  4  using  vehicle  decomposition  with 
dynamic  programming,  the  number  of  decisions  that  need  to 
be  evaluated  is 

R 


)-2]n  =  (9-3-2+4-2-2)-36  =  2520. 

1=1 

The  results  of  applying  the  two  approaches  and  their 
computational  requirements  are  displayed  in  the  following 
table.  The  algorithm  was  implemented  in  C  on  a  Pentium 
1.8  MHz  processor.  As  can  be  seen,  the  decomposition 
approach  obtains  a  schedule  with  a  value  that  is  within 
0.1%  of  the  optimal  solution  and  requires  approximately  6 
orders  of  magnitude  less  computation  time. 


Table  2  Computational  Results  for  Example  1 


Solution 

Approach 

Value 

Number  of 
Decisions 

Computation 

Time 

Exact  DP 

396.52 

90.7 

billion 

68  minutes  / 
4080  sec 

Decomposition 

396.22 

2520 

0.002  seconds 

VI.  Example  2 

The  next  example  is  motivated  by  a  problem  where  the 
main  objective  is  to  maintain  persistent  coverage  over 
several  geographic  areas  of  interest  to  ensure  that  time- 
sensitive  targets  can  be  attacked  within  the  required  time 
window. 

A.  Example  Details 

In  this  example,  the  objective  is  to  allocate  16  vehicles 
over  12  stages  to  4  areas  of  responsibility  in  a  manner  that 
maximizes  the  expected  value  of  targets  that  can  be 
destroyed.  The  vehicles  are  of  two  types  and  are 
distinguished  by  the  number  of  munitions  they  can  carry. 
The  munitions  are  of  two  types  and  are  distinguished  by 
their  probabilities  of  destroying  various  targets.  Table  3 
provides  the  availability  and  munitions  data  for  each  vehicle 
type. 


Table  3  Resource  Data  for  Example  2 


Number 

Max 

Avail 

Min 

Off-Time 

Type  1 
Weapons 

Type  2 
Weapons 

1 

8 

5 

3 

4 

2 

8 

5 

3 

2 

1 

The  targets  are  also  of  two  types:  high  value,  time- 
sensitive  targets  that  appear  during  each  stage  and  can  only 
be  destroyed  during  that  stage,  and  low  value,  non-time- 
sensitive  targets  that  appear  at  the  beginning  of  the  first 
stage  and  remain  until  they  are  destroyed.  The  value 
attained  during  a  particular  stage  was  the  expected  value  of 
destroyed  targets  if  the  allocated  munitions  were  optimally 
assigned  to  the  two  types  of  targets. 

Using  (1),  we  see  that  to  determine  the  number  of 
decisions  that  need  to  be  evaluated  to  obtain  the  dynamic 
programming  solution  is 

(4-5-P3)'®  (5)'®  •12  =  (115)‘'^  12  =  1.12-10”, 
which  is  too  large  to  compute.  We  instead  also  formulated 
the  problem  as  a  general  integer  programming  problem  and 
attempted  to  use  a  commercial  product  to  obtain  the  optimal 
solution.  Unfortunately,  it  was  also  unable  to  compute  the 
optimal  solution  after  many  hours  of  computation  time. 

Using  (2),  we  see  that  to  determine  the  number  of 


decisions  that  need  to  be  evaluated  to  obtain  a  solution 
using  vehicle  decomposition  with  dynamic  programming  is 
[(4-5-f3)-512]l6  =  22080. 

At  the  time  of  this  submission,  we  have  implemented  a 
vehicle  decomposition  approach  using  pruned  enumeration 
instead  of  dynamic  programming.  This  approach  compares 
a  subset  of  all  possibilities  to  solve  the  vehicle  sub¬ 
problems.  The  number  of  combinations  considered  was 
approximately  6000  for  each  vehicle,  yielding  a  total  of 
approximately  96000  total  evaluations.  The  possibilities 
that  were  pinned  from  the  entire  search  space  included 
those  where  vehicles  were  not  made  available  for  extended 
periods  of  time  and  those  in  which  vehicles  were  allowed  to 
move  from  one  area  of  responsibility  to  another  during  a 
particular  availability  period.  We  expect  that  the  solutions 
for  the  sub-problems  using  dynamic  programming  will  not 
vary  much  from  those  obtained  under  this  approach. 

The  vehicle  decomposition  approach  was  able  to  obtain 
an  approximate  solution  in  1 .3  seconds  that  yielded  a  higher 
expected  value  than  any  solution  obtained  using  the 
commercial  product  CPLEX  after  several  hours.  The  value 
obtained  as  a  function  of  time  is  illustrated  in  the  following 
figure.  We  expect  that  the  vehicle  decomposition  approach 
with  dynamic  programming  will  yield  similar  values  using 
less  computation  time. 
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Figure  4  Value  attained  as  a  function  of  time. 

vn.  Conclusion 

We  have  formulated  the  problem  of  allocating 
heterogeneous  resources  with  availability  constraints  over 
time  and  space  as  a  finite-state  decision  process  and 
proposed  a  decomposition  approach  using  dynamic 
programming  as  a  solution.  Our  approach  can  be  viewed  as 
an  approximate  dynamic  programming  technique  where  the 
value  that  could  be  attained  from  unassigned  resources  is 
set  to  0.  The  examples  provided  show  that  this  approach  is 
computationally  feasible  for  some  fairly  large  problems  and 
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that  it  obtains  solutions  that  are  very  close  to  the  optimal  for 
some  simple  value  functions.  Ongoing  work  includes 
extensions  involving  more  complex  value  fimctions  that 
may  require  assigning  approximate  values  to  unassigned 
resources  in  order  to  obtain  high  quality  solutions. 
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